We present a data-driven inference method that can synthesize aphotorealistic texture map of a complete 3D face model given a partial 2D viewof a person in the wild. After an initial estimation of shape and low-frequencyalbedo, we compute a high-frequency partial texture map, without the shadingcomponent, of the visible face area. To extract the fine appearance detailsfrom this incomplete input, we introduce a multi-scale detail analysistechnique based on mid-layer feature correlations extracted from a deepconvolutional neural network. We demonstrate that fitting a convex combinationof feature correlations from a high-resolution face database can yield asemantically plausible facial detail description of the entire face. A completeand photorealistic texture map can then be synthesized by iterativelyoptimizing for the reconstructed feature correlations. Using thesehigh-resolution textures and a commercial rendering framework, we can producehigh-fidelity 3D renderings that are visually comparable to those obtained withstate-of-the-art multi-view face capture systems. We demonstrate successfulface reconstructions from a wide range of low resolution input images,including those of historical figures. In addition to extensive evaluations, wevalidate the realism of our results using a crowdsourced user study.
展开▼
